1. Technical Field
This invention relates to systems based upon discrete probability distributions, and more particularly, to a method of tuning a decision network and a decision tree for use within a discrete probability distribution system.
2. Description of the Related Art
An increasing number of computer-based systems utilize models based upon discrete probability distributions. Such systems are capable of processing data and producing one or more possible discrete outputs. Discrete probability distribution systems stand in contrast to systems based upon continuous probability distributions. Examples of discrete probability distribution systems can include, but are in no way limited to, natural language understanding systems, speech processing systems, or any other system having an information processing capability which can produce one or more discrete outputs.
Discrete probability distribution systems can be modeled using a hierarchy of queries, also referred to as a decision tree model. The queries of the decision tree model can be directed to any of a variety of aspects concerning the data depending upon the particular domain of the data. Those skilled in the art will recognize that the actual queries, in addition to the placement of each query within the decision tree model, can significantly influence the effectiveness of the model. As a result, system designers constantly refine decision tree models in an effort to develop more robust systems.
Conventional methods of tuning or debugging decision tree models require repeated regression testing. For example, interpretations derived by the discrete probability distribution system can be compared to one or more known correct interpretations or truths. The deviation between the system determined interpretation and the truth can provide developers with insight as well as a measure of the accuracy of the particular decision tree model. Accordingly, the developer then can determine features of the processed data which likely caused the decision tree model to develop incorrect interpretations. Having identified one or more possible features, the developer can formulate a query to detect the identified feature or features. The query can be added to the decision tree model, which then must be retrained. In other words, the new query must be placed within the hierarchy of queries. After the new query is incorporated into the decision tree model, the model again can be tested.
Through continued regression testing, a developer can determine whether the added query improves, decreases, or makes no difference in the system performance. More experienced developers can gain an intuition for identifying features and formulating useful queries. Even experienced developers, however, can select ineffective, or possibly damaging queries. Regardless, each time a potential new query is identified and added to the decision tree model, the entire model must be retrained, statistically smoothed, and retested. Although the time required to complete one iteration of this procedure can vary depending upon the skill of the developer, the size of the test data set, and the processing power of the test machine, oftentimes an iteration can take a minimum of thirty to forty minutes and as long as several days or weeks in some cases.